Moving image editing device and method, and storage medium

ABSTRACT

A moving image editing device which uses a smart rendering scheme whereby the variation of playback time is minimized even when editing material data with a minimum time unit that does not match that of the save format. A material format which is a moving image format of the material data is acquired, and then it is determined whether or not the material format acquired by the material format acquisition unit matches a save format which is the moving image format to be saved. The material data is converted into the save format. The moving image data acquired by the format conversion unit is sequentially joined, and then the moving image data which is the save result is generated. The format conversion unit outputs a portion or the entirety of the material data itself regarding the material data determined to be matched by the format determination device, on the other hand, outputs data encoded into the save format after decoding a portion or the entirety of the material data regarding the material data determined not to be matched, and the joining unit repeats sequential joining so that the difference in the playback time with the material data may be minimized.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving image editing device andmethod, and a storage medium.

2. Description of the Related Art

Digital video cameras, and digital cameras with a function for shootingmoving images are becoming widely available, and opportunity to handlemoving image data in ordinary households and businesses is on the rise.Accordingly, an interest in moving image editing, which edits and savesmoving image data, is growing, and various moving image editingapplications are on the market.

Many moving image editing applications, which are typified byMicrosoft's Movie Maker™ and Adobe's Premiere™, provide a function forarbitrarily joining together a portion of a plurality of material datato generate a single moving image data. When the save data is generatedfrom a plurality of material data, such a moving image editingapplication achieves generation of save data by decoding all materialdata and re-encoding them into a predetermined moving image format.

However, since, in this edit/save method, all material data have to bedecoded and re-encoded, there are problems that it takes much time togenerate save data, and image quality is degraded every time editing isrepeated.

As techniques to solve these problems, sometimes an edit/save methodcalled a smart rendering is used. The smart rendering is a techniquethat, when a save format matches the moving image format of the materialdata, decoding and re-encoding are not performed, but the material datais used directly as save data.

According to this edit/save method, time required for decoding andre-encoding, which accounts for a large percentage of time required forsaving the result of editing, can be reduced. In addition, the smartrendering has the advantage that the image quality is not degraded evenif the same moving image signal is edited repeatedly because thematerial data does not have to be re-encoded.

As a moving image editing technique that uses a smart rendering, atechnique is described in Japanese Laid-Open Patent Publication (Kokai)No. 2004-104361. This prior art is a technique for applying a smartrendering even if there is material data whose format does not match thesave format in a moving image editing device. This prior art will bedescribed below with reference to FIG. 9.

FIG. 9 shows an overview of a moving image editing device that outputsmoving image data in an MPEG format.

In FIG. 9, the moving image editing device comprises an AVI importer(decoder) 902 and an MPEG importer 903, which are decoding devices fordecoding material data compressed and encoded in an AVI format and in anMoving Picture Experts Group (MPEG) format, respectively.

The moving image editing device also comprises a video editing controlprogram 901, which is a data editing device for arbitrarily joiningtogether a portion or all of a plurality of material data decoded bythese importers to generate edited edit data.

The moving image editing device also comprises an MPEG compiler(encoder) 904 for generating and outputting output moving image data inwhich the material data corresponding to the edit data edited by thevideo editing program 901 was encoded into a target MPEG format.

In this moving image editing device, when a plurality of partial datathat constitute the edit data have the same format as the target MPEGformat, the MPEG compiler 904 does not decode the material datacorresponding to the partial data, but uses them directly to generateoutput moving image data.

The above prior art achieves a sophisticated smart rendering with amethod that decodes only material data whose format does not match asave format, and converts obtained baseband data into the save format,then joins the material data to the remaining material data. In theabove prior art, a format indicates an encoding scheme such as aMotion-JPEG and an MPEG, a bit rate and an image size.

However, in the above prior art, the variation of playback time when thematerial data is converted into the save format has not been mentioned.Moving image data has a minimum time unit that can edit and save thedata. For example, the unit corresponds to a frame for a Motion-JPEG andGroup Of Picture (GOP) for an MPEG.

As described above, moving image data has different minimum time unitsdepending on frame rates and encoding schemes. Accordingly, when thematerial format and the save format are different, the format conversionrounds the playback time of the material data to the minimum time unitin the save format.

For example, a case is considered, in which a 29.97 fps Motion-JPEGserving as material data is converted as a 30 fps Motion-JPEG save data.In this case, since the frame rate of the save data is not an integralmultiple of the frame rate of the material data, the material data isrounded to 30 fps by conversion, therefore, a difference in the playbacktime arises.

Particularly, since when a large number of material data are edited andsaved into single save data, the difference arises for each materialdata, a problem arises, of differences accumulating in the save data,deviating considerably from the original playback time of the materialdata.

SUMMARY OF THE INVENTION

The present invention provides a moving image editing device and method,and a storage medium, which uses a smart rendering scheme whereby thevariation of playback time is minimized even when editing material datawith a minimum time unit that does not match that of the save format.

In a first aspect of the present invention, there is provided a movingimage editing device which edits and saves a plurality of moving imagedata as material data, comprising: a material format acquisition unitadapted to acquire a material format which is a moving image format ofthe material data; a format determination unit adapted to determinewhether or not the material format acquired by the material formatacquisition unit matches a save format which is the moving image formatto be saved; a format conversion unit adapted to convert the materialdata into the save format; and a joining unit adapted to sequentiallyjoin the moving image data acquired by the format conversion unit andgenerate the moving image data which is the save result, wherein theformat conversion unit is adapted to output a portion or the entirety ofthe material data itself regarding the material data determined to bematched by the format determination unit, on the other hand, output dataencoded into the save format after decoding a portion or the entirety ofthe material data regarding the material data determined not to bematched, and the joining unit is adapted to repeat sequential joining sothat the difference in the playback time with the material data may beminimized.

The moving image editing device can further comprises an effectinstruction unit adapted to give an instruction to add a video effectand an audio effect to any portion or the entirety of the material data,and the format conversion unit can decode the portion instructed by theeffect instruction unit regardless of the determination result by theformat determination unit, add the effect instructed by the effectinstruction unit, then encode the portion into the save format andoutputs the portion.

The joining unit can detect a difference between the playback time of aconversion result acquired by the format conversion unit and theplayback time of the material data when performing sequential joining,and select the joining portion of the conversion result so that thedifference is equal to or less than one frame duration in the saveformat.

The joining unit can detect a difference between the playback time of aconversion result acquired by the format conversion unit and theplayback time of the material data when performing sequential joining,and select the joining portion of the conversion result so that thedifference is equal to or less than one GOP duration in the save format.

In a second aspect of the present invention, there is provided a movingimage editing method which edits and saves a plurality of moving imagedata as material data, comprising: a material format acquisition step ofacquiring a material format which is a moving image format of thematerial data; a format determination step of determining whether or notthe material format acquired at the material format acquisition stepmatches a save format which is the moving image format to be saved; aformat conversion step of converting the material data into the saveformat; and a joining step of sequentially joining the moving image dataacquired at the format conversion step and generating the moving imagedata which is the save result, wherein the format conversion stepcomprises outputting a portion or the entirety of the material dataitself regarding the material data determined to be matched at theformat determination step, on the other hand, outputting data encodedinto the save format after decoding a portion or the entirety of thematerial data regarding the material data determined not to be matched,and the joining step comprises repeating sequential joining so that thedifference in the playback time with the material data may be minimized.

The moving image editing method can further comprises an effectinstruction step of giving an instruction to add a video effect and anaudio effect to any portion or the entirety of the material data, andthe format conversion step can comprise decoding the portion instructedby the effect instruction step regardless of the determination result bythe format determination step, adding the effect instructed by theeffect instruction step, then encoding the portion into the save formatand outputs the portion.

The joining step can comprise detecting a difference between theplayback time of a conversion result acquired at the format conversionstep and the playback time of the material data when performingsequential joining, and selecting the joining portion of the conversionresult so that the difference is equal to or less than one frameduration in the save format.

The joining step can comprise detecting a difference between theplayback time of a conversion result acquired at the format conversionstep and the playback time of the material data when performingsequential joining, and selecting the joining portion of the conversionresult so that the difference is equal to or less than one GOP durationin the save format.

In a third aspect of the present invention, there is provided a computerreadable storage medium storing a moving image editing program whichcauses a computer to execute a moving image editing method of editingand saving a plurality of material data as material data, wherein themoving image editing method comprises: a material format acquisitionstep of acquiring a material format, which is the moving image format ofthe material data; a format determination module step of determiningwhether or not the material format acquired at the material formatacquisition step matches the save format which is the moving imageformat to be saved; a format conversion step of converting the materialdata into the save format; and a joining step of sequentially joiningthe moving image data acquired at the format conversion module andgenerate the moving image data, which is the save result, wherein theformat conversion step comprises outputting a portion or the entirety ofthe material data itself regarding the material data determined to bematched at the format determination module, on the other hand,outputting data encoded into the save format after decoding a portion orthe entirety of the material data regarding the material data determinednot to be matched, and the joining module comprises repeating sequentialjoining so that the difference in the playback time with the materialdata may be minimized.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically showing the hardware configuration ofa moving image editing device according to an embodiment of the presentinvention.

FIG. 2 is a diagram schematically showing the function block of a movingimage editing device according to a first embodiment of the presentinvention.

FIG. 3 is a flowchart showing the procedure of video portion formatdetermination processing performed by a format determination unit inFIG. 2.

FIG. 4 is a flowchart showing the procedure of audio portion formatdetermination processing performed by the format determination unit inFIG. 2.

FIG. 5 is a first flowchart showing the procedure of video portionformat conversion processing performed by a format conversion unit inFIG. 2.

FIG. 6 is a flowchart showing the procedure of format joining processingperformed by a joining unit in FIG. 2.

FIG. 7 is a diagram schematically showing the function block of a movingimage editing device according to a second embodiment of the presentinvention.

FIG. 8 is a second flowchart showing the procedure of video portionformat conversion processing performed by the format conversion unit inFIG. 2.

FIG. 9 is a diagram schematically showing the configuration of a movingimage playback device related to a prior art example.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be descried in detail belowwith reference to the drawings.

The embodiments that will be described below is examples of a device forrealizing the present invention, and should be corrected or modified asappropriate depending on the configuration of a system to which thepresent invention is applied, and various conditions, therefore thepresent invention is not limited to the following embodiments.

FIG. 1 is a diagram schematically showing the hardware configuration ofa moving image editing device according to an embodiment of the presentinvention.

In FIG. 1, a moving image editing device is constituted by a personalcomputer (PC).

A central processing unit (CPU) 101 controls the operation ofediting/saving processing. A hard disk drive 102 records an executableprogram and digital data such as encoded moving image data in a readableformat. A program code and target image data recorded on the hard diskdrive 102 are loaded into a memory device 103.

An operating unit 104 is constituted by a keyboard, a mouse, or thelike, and is an input unit for a transfer operation. The input isdetected by the CPU 101, and notified to a program loaded into thememory device 103, then processing is performed.

A display unit (display) 105 is a CRT display, a liquid crystal panel,or the like, and displays the progress of transfer, and decoded movingimage signals. An external storage medium drive 106 is for reading anexternal storage medium such as a CD-ROM and a DVD-ROM. Each unitdescribed above is connected by an internal bus 107.

Although the present embodiment uses the moving image data recorded onthe hard disk drive 102, the moving image data may be executed throughthe external storage medium drive 106 when the moving image data isrecorded on the external storage medium. Similarly, the presentinvention is also applied to an embodiment in which the moving imagedata is loaded through a network.

In the present embodiment, a save data format is a Motion-JPEG, whichhas no inter-frame correlation. However, even in the save data format,which has inter-frame correlation, such as MPEG, the present inventionis applied by taking a frame in a Motion-JPEG as an editable and savableminimum time unit such as GOP.

FIG. 2 is a diagram schematically showing the function block of a movingimage editing device according to a first embodiment of the presentinvention.

In FIG. 2, two or more material data 201 to be edited can be entered.When two or more material data are entered, each material data 201 isentered successively. A material format acquisition unit 202 acquiresthe moving image format of the material data 201 material data bymaterial data.

The moving image format acquired by the material format acquisition unit202 is an encoding scheme, an image size, a frame rate, an audioencoding scheme, an audio bit rate, an audio channel number, and anaudio sampling rate.

A format determination unit 203 determines whether or not a materialformat acquired by the material format acquisition unit 202 and a saveformat completely match. A format conversion unit 204 refers to thedetermination result of the format determination unit 203, converts thematerial data 201 into the moving image data in the save format, andoutputs the data.

A joining unit 205 sequentially joins the moving image data outputtedfrom the format conversion unit 204 and generates save result data 206.

Next, the format determination unit 203 in FIG. 2 will be described withreference to FIGS. 3 and 4. The format determination unit 203 performsformat determination on each of a video portion and an audio portion.

Moreover, the flow of the format determination of one material data isdescribed in FIGS. 3 and 4, and when two or more material data arehandled, this processing is repeated by the number of material data.

FIG. 3 is a flowchart showing the procedure of video portion formatdetermination processing performed by a format determination unit inFIG. 2.

In FIG. 3, it is determined whether or not the encoding scheme of thevideo portion of the material data matches the save format in step S301.Similarly, in steps S302 and S303, regarding the video portion of thematerial data, it is determined whether or not the image size and theframe rate match the save format.

If a no-match is determined in any of steps S301, S302 and S303, theprocessing proceeds to step S305, and the video format is determined tobe a mismatch. On the contrary, if a match is determined in all ofsteps, the processing proceeds to step S304, and the video format isdetermined to be matched, followed terminating the processing.

FIG. 4 is a flowchart showing the procedure of audio portion formatdetermination processing performed by the format determination unit inFIG. 2.

In FIG. 4, whether or not the encoding scheme of the audio portion ofthe material data matches the save format is determined in step S401.Similarly, in steps S402, S403, and S404, regarding the audio portion ofthe material data, whether or not the bit rate, the number of channels,and the sampling rate match the save format is determined.

If a no-match is determined in any of steps S401, S402, S403 and S404,the processing proceeds to step S406, and the audio format is determinedto be a mismatch. On the contrary, if a match is determined in all ofsteps, the processing proceeds to step S405, and the audio format isdetermined to be matched. Then the processing is terminated.

Next, the format conversion unit 204 in FIG. 2 will be described. Theformat conversion unit 204 separately processes the video portion andthe audio portion of the material data according to the determinationresult of the format determination unit 203. Since the processing flowson the video portion and the audio portion are almost the same, only theprocessing on the video portion will be described in details withreference to FIG. 5.

In FIG. 5, a flow in which one of the material data is entered, andconverted into a save format is shown; when a plurality of material dataare handled, the processing is repeated by the number of material data.

FIG. 5 is a first flowchart showing the procedure of video portionformat conversion processing performed by the format conversion unit inFIG. 2.

In FIG. 5, in step 501, only a video portion is extracted from thematerial data. Successively, in step S502, the processing branches usingthe determination result of the format determination unit 203.

When it is determined that the format of the video portion of thematerial data does not match the save format, the processing proceeds tostep S503, otherwise, the processing proceeds to step S506.

In step S503, the data of the video portion extracted in step S501 isdecoded to generate baseband data. In step S504, the baseband datagenerated in step S503 is encoded so as to match the save format. Here,the types of formats to match are an encoding scheme, an image size, anda frame rate, as described above.

In the encoding in step S504, when encoding is performed with a framerate that is different from that of the material data, sometimes theminimum time unit may change, and moving image data with a differentplayback time than that of the material data may be generated. In thiscase, by generating moving image data in which the final frame isrepeated twice, moving image data having the playback time equal to orlonger than that of the material data is generated.

In step S505, the encoding result in step S504 is outputted as theresult of the format conversion. In addition, in step S506, the videoportion itself of the material data is outputted as the formatconversion result, followed by terminating the processing.

With the above procedure, when the format of the video portion of thematerial data matches the save format, the video portion itself of thematerial data can be outputted, and otherwise, decoding and encoding areperformed to generate and output the moving image data in the saveformat.

Moreover, in Steps S503 and S506, the processing can be performed on anyportion or the entirety of the material data. In addition, the sameprocessing is additionally performed on the audio portion. However,although the original minimum time unit of the audio portion is finerthan that of the video portion, processing is performed so as to matchthe minimum time unit of the save format of the video portion.

This allows the playback time to be the same for the audio portion andthe video portion, thus allowing the difference in the playback timewith the material data to be confined to less than one frame.

Next, the joining unit 205 in FIG. 2 will be described. The joining unit205 joins the format conversion results of the video portion and theaudio portion of each material data, which are outputted from the formatconversion unit 204, to generate save data.

As described above, since the playback time of the format conversionresult may have changed compared with the material data, the joiningunit 205 detects this difference and performs adjustments so that thedifference between the entirety of the save data and the entirety ofmaterial data becomes less than one frame.

FIG. 6 is a flowchart showing the procedure of format joining processingperformed by the joining unit in FIG. 2.

In FIG. 6, the save data is opened in step S601. In steps S602 and S603,the total playback time of the material data and the save data isinitialized to 0.

The total playback time is used to detect the difference between theplayback time of the material data and that of the save data generatedby the joining unit 205.

Steps S604 and S614 indicate that the processing of steps S605 to S613is repeatedly executed by the number of material data. In step S605, theplayback time of the material data is acquired. When only a portion ofthe material data is to be saved, the playback time of the portion to besaved is acquired.

In step S606, the playback time of the material data acquired in StepS605 is added to the total playback time of the material data. In stepS607, the playback time of the format conversion result outputted fromthe format conversion unit 204 is acquired. In step S608, the playbacktime of the format conversion result acquired in step S607 is added tothe total time of the save data.

In step S609, the total playback time of the material data updated instep S606 is compared with the total playback time of the save dataupdated in step S608. As a result of comparison, if the total playbacktime of the save data is determined to be one or more frame longer, theprocessing proceeds to step S610. On the contrary, when it is determinedthat the difference is less than one frame, the processing proceeds tostep S612.

In step S610, the total playback time of the save data is corrected soas to be equal to or longer than the total time of the material data,and so that the difference is less than one frame. Additionally, insubsequent Step S611, a portion to be joined is set in the formatconversion result.

Here, in order to match a target to be actually saved with the correctedcontent in step S610, in the format conversion result, the remaining ofthe subtraction of the playback time corrected in step S610 from the endportion is set as the portion to be joined.

In step S612, the entirety of the format conversion result is set as theportion to be joined.

In step S613, the portion to be joined, which was set in step S611 orS612, is saved as save data. Processing so far is performed repeatedlyby the number of material data, and finally, the save data is closed instep S615, followed by terminating the processing.

In the joining unit 205, performing the save processing according to theabove processing procedure allows the difference in playback time withthe material data to be confined to less than one frame, even if resultsfrom the format conversion of a plurality of material files are joined.

The above processing is performed sequentially, thus allowing the smartrendering to be provided whereby the variation of the playback time isminimized even when editing the material data with a minimum time unitthat does not match that of the save format.

Also, in a second embodiment of the present invention, the format of themoving image data to be edited is a Motion-JPEG, which has nointer-frame correlation.

FIG. 7 is a diagram schematically showing the function block of a movingimage editing device according to a second embodiment of the presentinvention.

In FIG. 7, two or more material data 701 to be edited can be entered.When two or more material data 701 are entered, each material data isentered successively. A material format acquisition unit 702 acquiresthe moving image format of the material data 701 material data bymaterial data. The moving image format acquired by the material formatacquisition unit 702 is an encoding scheme, an image size, a frame rate,an audio encoding scheme, an audio bit rate, an audio channel number,and an audio sampling rate.

An effect instruction unit 707 can give an instruction to add a videoeffect and an audio effect to any portion or the entirety of thematerial formats. Although this instruction is separately performed onthe video effect and the audio effect, both the minimum time units to beadded conform to the minimum time unit of the video portion. A videoeffect includes a rotational effect, a partial enlargement effect, andan adjustment effect such as brightness and contrast. An audio effectincludes fade-in and fade-out.

A format determination unit 703 determines whether or not a materialformat acquired by the material format acquisition unit 702 and a saveformat completely match, and whether or not the effect instruction unit707 gave an effect instruction. This determination is separatelyperformed on the video portion and the audio portion of the materialdata.

A format conversion unit 704 refers to the determination result of theformat determination unit 703 to reflect the video effect and the audioeffect instructed by the effect instruction unit 707 in the materialdata 701, which is then converted into the moving image in the saveformat and outputted. A joining unit 705 sequentially joins the movingimage data outputted from the format conversion unit 704 and generatessave result data 706.

The format determination unit 703 is for performing format determinationon each of the video portion and the audio portion, and the function andthe operation thereof are the same as those of the format determinationunit 203 described in the first embodiment.

Next, the format conversion unit 704 will be described. The formatconversion unit 704 separately processes the video portion and the audioportion of the material data according to the determination result ofthe format determination unit 703 and the instruction of the effectinstruction unit 707.

Since the processing flows on the video portion and the audio portionare almost the same, only the processing on the video portion will bedescribed in details with reference to FIG. 8. In FIG. 8, a flow inwhich one of the material data is entered, and converted into a saveformat is shown; when a plurality of material data are handled, theprocessing is repeated by the number of material data.

FIG. 8 is a second flowchart showing the procedure of video portionformat conversion processing performed by the format conversion unit inFIG. 2.

In FIG. 8, in step 801, only a video portion is extracted from thematerial data. Successively, in steps S802 to S812, the generation ofthe format conversion result is repeated according to the content of theeffect instruction by the effect instruction unit 707. In this repeat,format conversion is sequentially performed on each of portions of thematerial data temporally divided into an effect addition portion and anon-addition portion by the effect instruction unit 707.

For example, if an effect has been instructed by the effect instructionunit 707 on only the first half of some material data, the formatconversion unit 704 outputs the format conversion result in the firsthalf and second half order. In addition, if an effect has beeninstructed by the effect instruction unit 707 on only a portion in themiddle of some material data, the format conversion unit 704 outputs theformat conversion result in the first part, middle part, and last partorder.

In step S803, it is determined whether or not a video effect is added tothe entirety or a portion of the material data to be processed. When thevideo effect is added, the processing proceeds to step S806, otherwise,the processing proceeds to step S804.

In step S806, the data of the video portion of divided material data isdecoded to generate baseband data. In step S807, the video effect isreflected in the baseband data generated in step S806.

In step S808, the result reflecting the video effect is encoded so as tomatch the save format. Here, the types of formats to match are anencoding scheme, an image size, and a frame rate, as described above.

In the encoding in Step 808, depending on the save format, when encodingis performed with a frame rate that is different from that of thematerial data, sometimes the minimum time unit may change, and movingimage data with a different playback time than that of the material datamay be generated.

In this case, by generating moving image data in which the final frameis repeated twice, moving image data having the playback time equal toor longer than that of the material data is generated. In step S809, theencoding result generated in step S808 is outputted as the result offormat conversion. This is repeated by the number of divisions of thematerial data (step S813), followed by terminating the processing.

In step S804, the processing branches using the determination result ofthe format determination unit 703. When it is determined that the formatof the video portion of the material data matches the save format, theprocessing proceeds to step S805, otherwise, the processing proceeds tostep S810.

In step S810, the data of the video portion of divided material data isdecoded to generate baseband data. In step S811, the decoded result instep S810 is encoded so as to match the save format.

The processing in step S810 is the same as the encoding processing instep S808, and also adjusts the playback time as necessary. In stepS812, the encoding result generated in step S811 is outputted as theresult of format conversion. This is repeated by the number of divisionsof the material data (step S813), followed by terminating theprocessing.

In addition, in step S805, the video portion itself of the material datais outputted as the format conversion result. This is repeated by thenumber of divisions of the material data (Step S813), followed byterminating the processing.

With the above procedure, when the format of the video portion of thematerial data matches the save format, the video portion itself of thematerial data can be outputted, and otherwise, decoding and encoding canbe performed to generate and output the moving image data in the saveformat.

In addition, the added video effect can be reflected. Moreover, when thesimilar processing is additionally performed also on an audio portion,the difference in the playback time with the material data can beconfined to less than one frame.

However, although the minimum time unit of the audio portion is finerthan that of the video portion, the processing is performed so as tomatch the minimum time unit of the save format of the video portion,thus allowing the playback time to be the same for the audio portion andthe video portion.

The joining unit 705 joins the format conversion results of the videoportion and the audio portion of each material data, which are outputtedfrom the format conversion unit 704, to generate save data. The functionand the operation are the same as those of the joining unit 205described in the first embodiment.

The above processing is performed sequentially, thus allowing the smartrendering to be provided whereby the variation of the playback time isminimized even if the video effect or the audio effect is added by anedit task.

The moving image editing device of the present embodiment comprises amaterial format acquisition unit, which acquires a material format,which is the moving image format of the material data. In addition, themoving image editing device also comprises a format determination unit,which determines whether or not the material format acquired by thematerial format acquisition unit matches the save format, which is themoving image format to be saved. The moving image editing device alsocomprises a format conversion unit, which converts the material datainto the save format, and a joining unit, which sequentially joins themoving image data acquired by the format conversion unit to generatemoving image data, which is the save result.

Then the format conversion unit outputs a portion or the entirety of thematerial data itself regarding the material data determined to bematched by the format determination unit. Further, the format conversionunit outputs data encoded into save format after decoding a portion orthe entirety of the material data regarding the material data determinednot to be matched. The joining unit repeats sequential joining so thatthe difference in the playback time with the material data may beminimized.

The format conversion unit and the joining unit have the abovefunctions, thus allowing the smart rendering to be achieved whereby thevariation of the playback time is minimized even when editing thematerial data with a minimum time unit that does not match that of thesave format.

It is to be understood that the object of the present invention may alsobe accomplished by supplying a system or an apparatus with a storagemedium in which a program code of software which realizes the functionsof the above described embodiment is stored, and causing a computer (orCPU or MPU) of the system or apparatus to read out and execute theprogram code stored in the storage medium.

In this case, the program code itself read from the storage mediumrealizes the functions of any of the embodiments described above, andhence the program code and the storage medium in which the program codeis stored constitute the present invention.

Examples of the storage medium for supplying the program code include afloppy (registered trademark) disk, a hard disk, a magnetic-opticaldisk, a CD-ROM, a CD-R, a CD-RW, DVD-ROM, a DVD-RAM, a DVD-RW, a DVD+RW,a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively,the program may be downloaded via a network.

Further, it is to be understood that the functions of the abovedescribed embodiment may be accomplished not only by executing a programcode read out by a computer, but also by causing an OS (operatingsystem) or the like which operates on the computer to perform a part orall of the actual operations based on instructions of the program code.

Further, it is to be understood that the functions of the abovedescribed embodiment may be accomplished by writing a program code readout from the storage medium into a memory provided on an expansion boardinserted into a computer or in an expansion unit connected to thecomputer and then causing a CPU or the like provided in the expansionboard or the expansion unit to perform a part or all of the actualoperations based on instructions of the program code.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures and functions.

This application claims priority from Japanese Patent Application No.2006-233994 filed Aug. 30, 2006, which is hereby incorporated byreference herein in its entirety.

1. A moving image editing device which edits and saves a plurality ofmoving image data as material data, comprising: a material formatacquisition unit adapted to acquire a material format which is a movingimage format of the material data; a format determination unit adaptedto determine whether or not the material format acquired by the materialformat acquisition unit matches a save format which is the moving imageformat to be saved; a format conversion unit adapted to convert thematerial data into the save format; and a joining unit adapted tosequentially join the moving image data acquired by said formatconversion unit and generate the moving image data which is the saveresult, wherein said format conversion unit is adapted to output aportion or the entirety of the material data itself regarding thematerial data determined to be matched by said format determinationunit, on the other hand, output data encoded into the save format afterdecoding a portion or the entirety of the material data regarding thematerial data determined not to be matched, and said joining unit isadapted to repeat sequential joining so that the difference in theplayback time with the material data may be minimized.
 2. A moving imageediting device according to claim 1, comprising an effect instructionunit adapted to give an instruction to add a video effect and an audioeffect to any portion or the entirety of the material data, wherein saidformat conversion unit is adapted to decode the portion instructed bysaid effect instruction unit regardless of the determination result bysaid format determination unit, add the effect instructed by said effectinstruction unit, then encode the portion into the save format andoutputs the portion.
 3. A moving image editing device according to claim1 or 2, wherein said joining unit is adapted to detect a differencebetween the playback time of a conversion result acquired by said formatconversion unit and the playback time of the material data whenperforming sequential joining, and select the joining portion of theconversion result so that the difference is equal to or less than oneframe duration in the save format.
 4. A moving image editing deviceaccording to claim 1 or 2, wherein said joining unit is adapted todetect a difference between the playback time of a conversion resultacquired by said format conversion unit and the playback time of thematerial data when performing sequential joining, and select the joiningportion of the conversion result so that the difference is equal to orless than one GOP duration in the save format.
 5. A moving image editingmethod which edits and saves a plurality of moving image data asmaterial data, comprising: a material format acquisition step ofacquiring a material format which is a moving image format of thematerial data; a format determination step of determining whether or notthe material format acquired at the material format acquisition stepmatches a save format which is the moving image format to be saved; aformat conversion step of converting the material data into the saveformat; and a joining step of sequentially joining the moving image dataacquired at said format conversion step and generating the moving imagedata which is the save result, wherein said format conversion stepcomprises outputting a portion or the entirety of the material dataitself regarding the material data determined to be matched at saidformat determination step, on the other hand, outputting data encodedinto the save format after decoding a portion or the entirety of thematerial data regarding the material data determined not to be matched,and said joining step comprises repeating sequential joining so that thedifference in the playback time with the material data may be minimized.6. A moving image editing method according to claim 5, comprising aneffect instruction step of giving an instruction to add a video effectand an audio effect to any portion or the entirety of the material data,wherein said format conversion step comprises decoding the portioninstructed by said effect instruction step regardless of thedetermination result by said format determination step, adding theeffect instructed by said effect instruction step, then encoding theportion into the save format and outputs the portion.
 7. A moving imageediting method according to claim 5 or 6, wherein said joining stepcomprises detecting a difference between the playback time of aconversion result acquired at said format conversion step and theplayback time of the material data when performing sequential joining,and selecting the joining portion of the conversion result so that thedifference is equal to or less than one frame duration in the saveformat.
 8. A moving image editing method according to claim 5 or 6,wherein said joining step comprises detecting a difference between theplayback time of a conversion result acquired at said format conversionstep and the playback time of the material data when performingsequential joining, and selecting the joining portion of the conversionresult so that the difference is equal to or less than one GOP durationin the save format.
 9. A computer readable storage medium storing amoving image editing program which causes a computer to execute a movingimage editing method of editing and saving a plurality of material dataas material data, wherein said moving image editing method comprises: amaterial format acquisition step of acquiring a material format, whichis the moving image format of the material data; a format determinationmodule step of determining whether or not the material format acquiredat the material format acquisition step matches the save format which isthe moving image format to be saved; a format conversion step ofconverting the material data into the save format; and a joining step ofsequentially joining the moving image data acquired at said formatconversion module and generate the moving image data, which is the saveresult, wherein said format conversion step comprises outputting aportion or the entirety of the material data itself regarding thematerial data determined to be matched at said format determinationmodule, on the other hand, outputting data encoded into the save formatafter decoding a portion or the entirety of the material data regardingthe material data determined not to be matched, and said joining modulecomprises repeating sequential joining so that the difference in theplayback time with the material data may be minimized.